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ABSTRACT 

Using nationally representative data generated by the 
normal, day-to-day operation of schools, this research tasted the 
major conclusions drawn from case studies focusing on principal 
effectiveness. On the basis of detailed information regarding 
individual student characteristics and achievements, detailed 
characteristics of teachers and their instructional choices and 
methods, and detailed information on the characteristics and behavior 
of principals, relevant changes in the educational process and in 
student outcomes were measured in relation to particular types of 
principal behavior. After the introduction, section 2 reviews the 
literature on the principal's role in student achievement. In section 
3, a general contextual model for student achievement is presented, 
and tha research methodology is described in detail, including the 
educational production function approach to the model and the 
specification of tha variables for principal behavior and attributes. 
The empirical results are presented in section 4. The study concludes 
that principals directly affect student achievement through providing 
strong leadership and reducing conflicts among the participants in 
tha educational process. Retarences are included, along with an 
appendix listing variables used in the analysis. (TE) 
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Principal Ef ftctivsnaas: Using Nonaxpariaantal Data to Assess tha 

Findings of Cass Studiss 

Randall W. Eberts 
and 
Joe A. Stone 

I. Introduction 

The literature on effective schools and administrative leadership 
paints to a variety of determinants of student achievement that can be 
v. (influenced by various types of administrative behavior. The major studies 

I rrom which such conclusions are drawn include Brookover et al. (1979). 

\s^^Jutter et al. (1979), Ayrault and Crosetto (1982), Edmonds (1979), Goodlad et 
al. (1979), Duke, Showers, and Imber (1980), Wellisch et al. (1978), Persell 
(1982), Greenfield (1982), and Gall (1983). Some of these studies are 
conveniently summarized by D'Amico (1982) and DeBevoise (1984). 

Two major channels through which administrators affect student 
achievement are suggested by the literature: (1) through the design, 
coordination, implementation, and evaluation of instructional programs, and 
(2) through modification of teacher behavior. A third channel articulated by 
some researchers, although not as clearly defined as the previous two, is the 
general climate set by administrators %n their interactions with teachers and 
students. Figure 1 provides a simple illustration of the three channels and 
other intervening factors. 

Specific types of principal behavior identified with effective 
schools in these studies Include: setting clear priorities and objectives 
that emphasize basic skill aco.uisition, assuming responsibility for 
evaluations of the achievement of these objectives, organizing and 
participating in staff development and inservice training programs, being a 
consistent, assertive disciplinarian, and working with teachers to achieve a 

1 
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Figure 1 Channels Through Which Principals Affect 
Student Achievement Gains 



Program 




(not significantly 
affected by principal 
bshavior) 



consensus on objectives, methods, and staff development programs. While 
there is widespread agreement regarding these traits of principals, there is 
disagreement regarding whether principals should provide direct supervision 
of teachers on instructional matters* 

Unfortunately, virtually all the conclusions regarding principal 
effectiveness are based upon individual case studies or limited data sets. 
Moreover, many of these studies examine only the determinants and process of 
student learning, rather than carrying the inquiry a step further to examine 
how these factors actually affect student achievement. Case studies (induced 
experiments) have many advantages in generating hypotheses, in evaluating the 
implementation of new techniques, and in providing detailed explanations and 
backgrounds for observed phenomena. A deficiency of case studies, however, 
is that their representativeness can rarely be demonstrated. 

Using nationally representative data generated by the normal, 
day-to-day operations of schools, (i.e., data not obtained from special 
experiments), we tested the major conclusions drawn from case studies 
regarding principal effectiveness. We measured relevant changes in the 
educational process and in student outcomes attributable to particular types 
of principal behavior. Although secondary analysis of data has its own set 
of pitfalls ^we believe the results of our research both extend and 
complement previous case studies. With detailed information regarding 
individual student characteristics and achievements (including pre- and 
post-test scores on standardized mathematics achievement tests), detailed 
characteristics of teachers and their instructional choices and methods, and 
detailed information on the characteristics and behavior of principals, we 
assessed under ceteris paribus conditions many of the standing conclusions 
regarding principal effectiveness. 

In addition, the hierarchical nature of the data (e.g., student, 
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classroom, teacher, school, and district levels) provided opportunities to 

explore both the direct and indirect effects of principal behavior on student 

achievement. Thus, by being able to match students with teachers and 

4 

teachers with principals, we were able to explore the various paths through 
which effective principal behavior is transmitted to students* Finally, by 
examining the effects of principal characteristics on students in over 300 
schools nationwide, we assessed whether the findings obtained from previous 
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studies hold true in larger sampled* V 

The paper is organized as follows* In Section II we review in more 
detail the literature on the principal's role in student achievement* In 
Section III we offer a general contextual model for student achievement, 
discuss the educational production function approach to the model, and 
explain in detail the specification of the variables for principal behavior 
and attributes* In Section IV we present and discuss the empirical results* 
A final section provides a brief summery of our major conclusions* 

II* Literature on Effective Principal Behavior 

* 

The literature on effective principal behavior addresses two broad 

j 

issues: (1) do principals affect student achievement? and (2) if j so, what 
elements of principal behavior are effective? A recent synthesis of the 
literature by DeBevoise (1984) and another by D'Amico (1982) offer good 
reviews of the literature* The review we present here is intended to 
identify a number of measures of principal behavior and to extract several 
hypotheses about principal effectiveness to be used in the analysis reported 
later in the paper* 

Most of the studies that address the issue of effective principal 
behavior concentrate on the principal as instructional leader* For example, 
Wellisch and others, in evaluating the effect of various attributes of 
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administrators on student achievement , suggest that three characteristics of 
administrative behavior are important: how strongly administrators feel 
^ about instruction, whether they communicate their ideas concerning 

instruction, and the extent to w^ich they assume responsibility for 
instruction (1978, p. 215) • A number of studies, including Wellisch and 
others, have provided evidence that administrative leadership is indeed a 
promising area for research related to school improvement* Keeler and 
Andrews (1972), for example, find that the leadership behavior of principals, 
as perceived by their staffs, was significantly related to the productivity 
of schools* More recently, a number of other researchers have provided 
corroborating evidence in support of the hypothesis that school principal 
Involvement in instructional leadership is correlated with improved student 
outcomes (Edmonds 1979, Brookover et al« 1979, and Wellisch and others, 
1978)* In addition, Wellisch and others contend that principals in schools 
where there had Seen student achievement gains were significantly more likely 
to "review and discuss teaching performance regularly with their staff" (p. 
217). They also report that principals and teachers in these more successful 
schools were significantly more likely to report a high degree of program 

coordination. 

* 

Currently there is much debate about the potential of administrative 
leadership as a key to Increase student achievement. WKile the studies noted 
above support the notion that principal Involvement in instructional 
leadership will lead to scliool^Lmprovement, others have informed us that 
principals who actively engage in such activities are Indeed rare (Deal 1975; 
Cohen and Miller 1980). Moreover, even researchers who accept the notion 
that instructional leadership is linked to school improvement have asserted 
that it is not the principal who is important per se, but rather that there 
are critical support functions that roust be carried out* These support 
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functions, may be performed by a variety of school personnel other than the 
principal including curriculum specialists, department heads, and teachers 
(Gersten and Carnine 1981). Finally, yet others caution that even when 
principals engage in the comprehensive set of| tasks referred to as 
instructional leadership, the participation ok teachers must also be 
considered as a critical variable (Wellisch and others, 1978). 
Unfortunately,, Wellisch and others do not include a measure of the 
participation of teachers in their study of "School Management and 
Organization in Successful Schools." 

Second, if administrative leadership is important to student 
achievement, what elements of administrative behavior are most important? 
Although an important role of the principal is to provide instructional 
leadership, very little of the principal's time is spent in any instructional 
interaction with students. The time the principal does spend with students 
is either related to disciplinary matters or to observing teachers in the 
classroom. The effect of principals on student achievement comes primarily 
through various interactions with teachers. The potential effect of this 
interaction can best be understood by considering what the ideal role of a 
principal should be. Edmonds, Cohen, Brookover, Gersten and Carnine, to 
mention a few, identify a number of ways in which the principal can enhance 
educational programs. These elements include 1) maintaining order, 2) acting 
as aa agent of change, 3) setting clear objectives, 4) conveying high 
expectations for student achievement, 5) offering support and guidance to 
teachers, 6) providing public rewards and incentives, and 7) spending time in 
the class rooo. 

These activities have not yet been entered in any systematic way into 
educational production functions in order to estimate their effect on student 
outcomes. Fortunately, the data set used in this analysis has a number of 
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variables which relate to these attributes. The principal's instructional 

w 

leadership can be captured In part by variables reflecting 1) the time 
principals spend In curriculum development, 2) the degree to which teachers 
sense their school has a coordinated curriculum, 3) the time principals spend 
related to needs assessment, problem planning, and program evaluation, 4) 
frequency with which the principal engages in classroom observation, 5) the 
degree to which teachers perceive that the principal is supportive of them, 
and 6) the degree to which teachers perceive the principal is supportive of 
innovative instructional practices. 

III. Research Design and Methodology 

The CEPM Research Paradigm of the determinants of student achievement 
gains, reproduced in Figure 2, provides a stylised summary of the way various 
researchers view the educational process* Our study focused on features of 
the educational process pertinent to administrative leadership* In 
particular, we examined the effect of principal behavior on achievement gains 
through the organization and implementation of instructional activities, 
through modifications of teacher characteristics and behavior, and through 
othei changes in the school climate. 

In previous work (Eberts and Stone 1984), we looked at the effect of 

work characteristics on teacher activities, including teacher time allocation 

o 

and student time on task* In turn, we linked these activities directly to 
student achievement gains* In relation to Figure 2, we have looked at the 
relationships outlined from work characteristics (e*g*, work agenda, work 
resources, and work incentives) down "to student achievement* Among the most 

7 

important determinates of student achievement gains explored in the previous 
study was the time teachers spend in instruction and preparation. 
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Figure 2 CEPM Research Paradigm 
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To focus on the channels through which principals affect student 
achievement, as previously outlined, models of the educational process must 
be specified and tested in greater detail, capturing more subtle features of 

to 

the process related to principal effectiveness, tfhus, with substantial work 
already completed analyzing the factors outlined in the lower half of Figure 
2, we now turn to some of the most prominent hypotheses regarding factors one 
level higher (Administration and Organization) related to principal behavior, 

and to how fhose are transmitted to student achievement. This ambitious task 

\ 

was moderated substantially by the fact that we focused 'pn a limited number 
of hypotheses, those suggested by previous research as most dominant. In the 
remainder of. this section, we first consider the education production 
function methodology we employed. Next, we describe the data used in our 
empirical tests and describe in some detail the variables used (especially, 
those related to principal effectiveness). Finally, we explain how the 
variables for principal behavior and effectiveness were entered into the 
educational production function. 

Educational Production Functions 

Education is a service that takes students, with whatever attributes 
they bring to the classroom, and transforms them into humans with different 
qualities* Educational production functions relate differences in the 
quality of students to differences in school resources they received. 
Educational production functions, therefore, are a convenient vehicle for 
exploring the channels through which principals may affect student 
achievement. Although, specifications of educational production functions 
dlfcfer among studies, most models share the features described by equation 
(1), which is borrowed from Hanushek (1979): 



< l > A it - f(B it ,Pit.S it , l ± ) 

where 

A it - student, outcomes of ith students at time t, 

B it ■ vector of family background influences of ith student 
cumulative to time t, • 

P it ■ vector of influence of peers of ith student cumulative 

to time t, ^ 
% 

S it - vector of school inputs of ith student cumulative to 

time t, and I . vector of innate abilities of ith 
student. 

The model incorporates a number of essential aspects of the 
educational process* First , inputs are those that are relevant to the 
individual student. Second, the inputs are cumulative which reflects the 
fact that schooling and other experiences in past years have a bearing on 
student outcomes in the present period* Third, school inputs include 
purchased (e.g., teachers) as well as non-purchased inputs (e.g., peer 
groups). Fourth, the allocation of resources is predetermined from the 
perspective of the production function. 

A somewhat popular variant of the model and one which requires 
substantially less data collection, is the value added model. Instead of 
considering the contribution of past inputs on student outcomes, this 
specification considers che changes in student outcomes between two time 
periods, usually the beginning and end of a particular school year. This 
formulation reduces the data re juirements since' inputs are only collected 
over the same two year period, not over an extended period of time. The 
value added model results from simply subtracting equation (1) for period 
t* from equation (1) for period t, thru yielding 

(2) A u - t*[^ i (t-t*),P i (t-t*),S i (t-t*),I i ,A u *] 

Student outcomes in the earlier period (A it *) may be considered 
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pretests taken by students at the beginning of the school ye, r. These scores 
are then compared with scores of tests taken at the end of the school year. 
In this way, the gains in student outcomes attributed to a flow of 
educational services within a given time period can be assessed. Variables 
such as teache retime and student time on task are used to capture the flow of 
resources. 

In most of the studies considered under the rubric of educational 
production functions, standardized test scores of cognitive skills are used 
as the measure of student outcomes* Test scores are obviously not intended 
to measure all the attributes of education* School outcomes encompass, in 
addition to the acquisition of skills, conveyance of social norms, 
development of creative skills, aid the provision of custodial services* 
A few studies have considered student attributes other than test 
scores as dependent variables* For example, Levin (1970), and Boardman, 
Davis, and Sanday (1977) considered student attitudes; Katzman (1971) looked 
at attendance rates; and Katzman (1971) and Burkhead, Fox, and Holland (1967) 
jdsed college continuation and dropout rates* These are all sensible 
X* measures. The decision of the vast majority of studies to use cognitive test 
scores results from a combination of availability and a certain 
conceptualization of education* Most school districts administer some form 

of standardized tests* Even though there is considerable controversy over 

# 

what these tests actually measure, educators tend to believe that they ars 
Important. Performance on tests is used to advance students through the 
educational system, evaluate programs, and even to allocate funds. Further, 
it appears, given the recent concern over declining SAT scores, that Interest 
In test scores is Increasing. 
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Data 

Nonexperimental data provide a snapshot of the workings of school 
systems. Thus, analysis of those data can be used to show how all the 
various educational resources, institutional policies, and behavioral 
responses of teachers, principals, and students come to bear on the 
achievement gains of students. Consequently, this approach to assessing the 
effectiveness of principals requires a substantial data base in oider to 
capture many of the important mechanisms at work in schools. Basically, 
three groups of data are required. First, information Is needed regarding 
the educational process. This would include relevant student demographics, 
teacher characteristics and tasks performed, and classroom organization. 
Second, information is required about principals. In particular, we need to 
know their characteristics, their involvement in leadership activities, and 
how they interact with teachers and students. Third, variables reflecting 
the institutional structure of the school and district are important to 
behavioral responses of teachers, students, and principals to these policies. 

Fortunately, the data needed to undertake our study had already been • 
compiled from a study sponsored in the mid-1970s by the Office of Education 
and conducted by the Systems Development Corporation (SDC). They amassed 
data from over 100,000 students in grades three through six in over 300 
districts selected to be representative of schools across the country. They 
followed the same group of students for three years and recorded the amount 
of resources each student received at the classroom level, school level, and 
district level. They gathered student background information, teacher and 
principal characteristics, and detailed information about school and district 
decision-making processes. 

Two general types of data were gathered regarding principal 



instructional leadership. The first set of data records the amount of time 
principals spend in activities related to math curriculum development and to 
assessing needs, planning instructional programs , and evaluating these 
programs* The second set of data reflects teachers' ani principal 
assessments of the effectiveness of certain leadership activities and how 
well the staff works together. Teachers and principals were asked, for 
example, if tfiey strongly agree, agree, disagree, or strongly disagree with 
the following questions: 

1. School programs are well planned and clear; % 

2. Principal provides active leadership to math and reading programs; 

3. Teachers in this school work well together; 

4. Administrators keep teachers well-informed; 

5. Conflicts among individuals are identified and faced, and not allowed 
to fester. 

By recording responses to these questions from both teachers and 
administrators, it is possible to check whether an individual's own 
assessment of his or her actions is more effective than another person's 
assessment. A more detailed description of the SDC data is attached as 
Appendix A. 

Specification of Principal Behavior and Effectiveness 

The variables describing principal characteristics and activities 
were entered into the educational production function in two ways. First, 
those variables that relate to the direct effects of administrative behavior 
or to indirect effects that are not explicitly specified (e.g., teacher's 
assessment of certain leadership activities and characteristics) were entered 
directly into the educational production function. Second, those principal 
attributes and behaviors that affect student achievement indirectly through 
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their effect on teacher behavior were used to explain differences across 
classrooms in teacher activities and classroom structure* 

For the direct principal effects and those indirect effects that were 
unobserved, we constructed four variables in addition to such principal 
attributes as highest degree, experience in teaching, and eflfcferience in 
administration. In each case we were able to construct measures that 
reflected the Joint perceptions of teachers and principals, as well as the 
degree to which the teachers' and principals' perceptions differ. 

The four variables are LEAD, INSTR, CONFL and FACE* The LEAD 
represents the average perception of both teachers and the principal of 
whether the principal exhibits "active leadership." The perceptions were 
Joint in the sense that principal responses were averaged and then 
transformed into unit-normal variates* The second variable, INSTR, is the 
Joint perception of the principal's involvement in the math curriculum and 
whether he is an active participant in teacher inservice programs. CONFL is 
a composite variable reflecting the Joint perceptions of whether teachers are 
satisfied with the principals decisions and whether the principal is 
effective in identifying conflicts. \FACE reflects perceptions of whether thte 



achievement* 

Perceptions do not always coincide, however* Thus we included for 
each of the four variables above a corresponding variable that reflects the 
degree of disagreement between teachers and the principal in each area* To 
construct each of these we subtracted the teacher's perception from the 
principal's and multiplied the difference by the absolute value of the 
difference* This is akin to squaring the difference, except that the sign is 
retained* Larger differences are, therefore, assigned to more than 



principal 




All four variables have been 



suggested in the literature as areas w^ere principals can affect student 
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proportionate effects. Divergences of opinion, where the principal perceives 
the situation more favorably than teachers, will be associated with less 

effective schooling and lower student achievement. Constructing the 

« 

variables in this way follows from the understanding that principal 
effectiveness is transmitted to students through teachers since principals 
spend relatively little time in contact with students. Consequently, 
teachers 49 perceptions of the performance of principals and thus the 
divergence of opinion of teachers and principals about the principal's 
performance are expected to influence student achievement in the direction 
specified. The names of these "agreement" variables are ALE AD, AINSTR, 
ACONFL, and AFACE, respectively. 

We were also interested in whether other principal attributes aro 
important. To pursue this issue we included in the analysis the highest 
degree held by the principal, the principal's experience in teaching, and the 
principal's experience in administration^^All were hypothesized to enter 
with positive signs. 

To explore the indirect effects of principals on student achievement 
in detail we considered the influence of the principal variables above on 
more proximate determinants of achievement. We were particularly interested 
in whether principals are able to affect such things as teacher time in 
instruction, teacher time in preparation, teacher time in administration, and 
math in-service programs. To do this we examined each of these separately at 
the teacher or classroom level. 

Specification of Other Control Variables 

In addition to the variables that were of central interest, we also 
included a number of control variables for various background characteristics 
of students, teachers, and schools. For the student-level data these 

15 
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included the sex of the student, rt>ce, the pretest score^^ndthVvp^retest 
score squared, to account for nonllnearlties) , student socioeconomic status, 
school climate, the teacher's highest degree, the teacher's experience, the 
number of math courses taken by the teacher since the degree, various 
staffing ratios (administrators per student and clerical workers per 
student), the school's average daily attendance, and whether the teachers are 
covered by a collective bargaining agreement. We had little direct interest 
in these variables, hence offer little discussion or interpretation of their 
empirical results here, but those who are interested may refer to chapter 
three of Eberts and Stone (1984). 

IV. Empirical Results 

Direct and Unobserved Indirect Effects 

The means and standard deviations of the variables are displayed in 
Table 1. The student-level estimates used to test our hypotheses regarding 
the direct and unobserved indirect effects of principals on student 

achievement gains are presented in Table 2 and summarized in Table 5. The 

* 

coefficient for two of the four variables regarding perceptions of the 
principal's behavior (INSTR and CONFL) are significantly positive at the 0.05 
level, indicating that instructional leadership and conflict resolution are 
effective principal traits. The coefficients for perceptions of how well the 
teachers and principal work together (FACE) is not significantly related to 
student achievement, and the coefficient for perceptions of the principal as 
an active leader (aside from instruction and conflict resolution) is negative 
(significantly so at the 0.05 level). The coefficients for principal 
attributes of teaching experience and administrative experience are both 
significantly positive at the 0.05 level, but the coefficient for the 
principal's highest degree is significantly negative, perhaps indicating that 
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such principals are assigned to more difficult schools. The negative 
relationship between degree level and student achievement is also found for 
teachers in this and many others studies. Murnane (1981) interprets this 
finding to reflect the propensity of school personnel to obtain additional 
education and as a way to advance al>ng the salary schedule than as a way to 
improve teaching skills. 

The perception disagreement variables did not generally enter 
Significantly. The coefficient for ACONFL, however, was significantly 
negative as expected. This coefficient, taken along with the significantly 
positive coefficient for CONFL, suggested that the conflict resolution role 
of the principal, and perceptions of the principal's performance in this 
role, are a preeminent part of effective principal behavior. Moreover, 
active leadership in noninstructional areas (or areas of little conflict) and 
working well together in areas of little conflict appear to be unrelated to 
student performance. 

The background control variables generally entered as expected, with 
teacher's highest degree and math inservice programs important exceptions. 
The previous sign for teacher's highest degree has already been discussed. 
The perverse sign for the coefficient for math inservice programs probably 
indicates that such programs "signal" difficulties with student math 
performance. 

Indirect Effects 

The teacher or classroom level estimates used to test our hypotheses 
regarding the indirect effects of principals on teacher time in instruction, 
in preparation, in administration, and on math in-service programs are 
presented in Table 4 and summarized in Table 5. Again, for each equation we 
regress the teacher time variable on the principal variables and attributes, 
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as well as on other background control variables. All these variables, of 
course, are now at the teacher or classroom level. 

The results for teacher time In Instruction revealed only one 
significant coefficient for the principal variables, tTrntfjEor differences in 
perceptions regarding conflict identification and resolution (ACONFL). This 
coefficient entered significantly negative as predicted and again underscores 
the importance of this principal role. 

The results for teacher time in preparation revealed two significant 
coefficients with the predicted signs— those for LEAD and CONFL. Both enter 
significantly with positive signs. The coefficient for FACE, perceptions of 
how well the teachers and principal work together, was significantly 
negative, counter to predictions. (One could speculate that 
principal- teacher cooperation reduces the importance of k ditional 
preparation time.) None of the remaining principal variables entered 
significantly at the 0.05 level. 

Not surprisingly, perhaps, teacher time in administration is 
invariant with respect to any of the principal variables. None of these 
estimates were significant. The results of math in-service programs, 
however, were strikingly different. The coefficients for both INSTR and 
CONFL, the perceptions of instructional leadership and conflict resolution, 
were significantly positive. In addition, the coefficent for ACONFL, the 
variable for the difference in perceptions of conflict resolution, was 
significantly negative. This again suggests the importance of this role for 
effective principals. 

In summary, we generally found for both the direct, unobserved 
indirect, and indirect paths that instructional leadership and conflict 
resolution are important roles for effective principals, and that divergence 
of opinion regarding the resolution of conflicts is associated with less 
effective schools. Thus, the results are generally supportive of the central 
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thrust of previous findings from case studies. We did find, however, that 
controlling for instructional leadership and conflict resolution, active 
leadership, and working well together in other areas are not associated with 
effective schools or student achievement. The latter may be an affirmation 
of T. Sizer's (See DeBevoise 1984, p. 17) speculation that a "strong 
principal" does not necessarily make an effective school. 

V. Caveats and Conclusions 

One aspect of the analysis that needs to be addressed is the 
contribution of principals on student achievement. As mentioned earlier, 
seven out of the eleven variables included in the student achievement 
equation were statistically significantly different from zero. This 
indicates that the variables have significant independent effects on student 
test scores. However, when one considers the percentage of the variation in 
student test scores explained by the principal variables, the magnitude is 
very small, only 0.4 percent. At first, it may seejji as if the effect of 
principal variables are too trivial to be worth discussing. But given the 
nature of the analysis, this is not necessarily so. It must be recognized 
that the student achievement model is capturing the effect of principals 
during one school year, which is a very short time period considering the 
student is in school for at least twelve years. 

The cumulative nature of the process is clearly evident by comparing 
the R-squared of the student achievement model when the pretest score is 
included and when it is omitted. When the pretest score is removed the 
percentage of the variation in student test scores explained by the remaining 
variables drops from 55 percent to 25 percent. Obviously, the pretest score, 
which captures the effect of schooling and other influences in previous 
years, explains a large proportion of the variation in student test scores. 
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Viewed in this context, the magnitude of the principal variables is no less 
significant than many of the teacher-related variables* In fact, comparing 
the standardized beta coefficients (not reported here) of the student 
achievement model, the principal variable, the time principals spend 
assessing the math program, has the largest single effect on student 
achievement of all the school-based impact, including teacher time in 
instruction* 

1 The relatively small explanatory power of the school-based inputs \ 

included in the student achievement equation is in many respects reassuring, ) 
particularly when compared to other research conducted in this area. If 
principals (and teachers) have a much larger effect on student achievement 
than estimated here, we would expect to find tremendously significant effects 
recorded in multivariate- type analyses as well as case studies* If such,, 
large effects actually existed, then there should be no debate on what makes 
for an effective principal (or teacher); the prescription would be clear from 
any properly performed analysis* 

Instead of uncovering strong relationships between principals and 
student achievement, researchers have uncovered at best only subtle 
characteristics that distinguish a principal who is effective from one who is 
not effective* Indeed, our analysis does no more and no less than previous 
studies on principal effectiveness, with one exception* Taking the 
hypotheses that have been set forth from case studies which have examined a 
few principals in a handful of schools, we extended the analysis to test 
these hypotheses for a representative sample of students and principals 
across the country* We found that principals do make a difference in student 
achievement, but, like the findings of previous studies, the attributes that 
are effective are subtle, with a relatively small impact recorded over a 
single year* 
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Our conclusion, therefore, closely parallels the conclusion drawn in 

a recent synthesis of the research on Principal Leadership: 

Ultimately the provision of instructional leadership can be 
viewed as a responsibility that is shared by a community of 
people both within and outside the school. Principals 
initiate, encourage, and facilitate the accomplishment of 
instructional improvement according to their own abilities, 
styles, and contextual circumstances. They still need a. lot 
of help from others if improvement is to become noruu" 
(DeBevoise 1984, Educational Leadership, p. 20)* 

We might add that principals appear to be equal partners with 

teachers in their contribution to student achievement* Principals directly 

affect students through providing strong leadership and reducing conflicts 

among the participants in the educational process. 
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Table 1: Means of Variables at the Student Level 



Variable 
Name 


_ _ • mm mm. f • « ♦ 

Variable Definition 


Mean 


Standard 
Deviation 


LEAD 


Active Principal Leadership 


•001 


.733 


INSTR 


Instructional leadership r > 


•002 


.654 


OCNFL 


Conflict identified 


-.002 


•594 


FACE 


Work well together 


-•013 


1.534 


ALEAD 


absolute value of LEAD 


•070 


3.135 


AINSTR 


absolute value of INSTR 


• 198 


2.972 


AOCNFL 


Absolute value of OONFL 


-.001 


2.975 


APACE 


Absolute value of FACE 


•030 


3.393 


PDBGREE 


Highest degree of Principal 


2.997 


• 214 


MM mm m fm 

PRINEXP 


Experience teaching of Principal 


10.189 


5.355 


PFYPAD 


Experience in administration 


15.078 


7.541 


SEX 


Female equals one 


.514 


.582 


RACE 


white equals one 


.731 


.108 


SB0034 


Childhood Experience 


1.061 


1.160 


SBC035 


Total parental involvement 


1.879 


1.683 


SBC037 


Economic status 


224.774 


89.547 


PRE 


Protect score 


29.836 


35.531 


PRESQ 


Pretes *. score squared 


2152.646 


M m am f%0m^ 

35137.926 


ADS1UD 


^L*J m^mm tm ^mMm^m MANAMA m mmmA^m * ^Bfe.^^^^ mm^mmm^^Lm *W Am**. 

Adniinlstrators/ student ratio 


• UU4 


• QUI 


\JCCOL\lU 




019 


.013 


TTINSTR 


Teacher's time in instruction 


4.886 


.766 


TTPREP 


Teacher's time in preparation 


1.404 


.725 


TTACMIN 


Teacher's time in administration 


.790 


.627 


TQA001 


Years teaching of teacher 


11.981 


7.840 


TQA004 


Highest degree of teacher 


2.465 


.518 


TQW0O6 


Math courses taken by teacher 


.610 


1.078 


TQA012 


Math inservice 


7.368 


14.047 


SCHADA 


School enrollment 


553.232 


296.162 


UNION 


Union equals one 


.636 


.481 


POST 


Poet test score 


38.894 


13.005 



Source: "Sustaining Effects Study" ccncl 
Corporation, Santa Monica, GA. 
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Table 2: Estimates of an Educational Production Function 

with Principal Variables 



Variable 


Variable Definition 


Estimate 


T-ratio 


NdlM 








intercept. 


Active principal Leadership 


6.721 


4.67 


LtAU 


-.216 


1*96 


DJSTR 


Instructional Leadership 


• 247 


O. AA 

2e20 


vJUNrlj 


Conflict identified 


• 548 


3*14 


FACE 


Vtork well together 


-•082 


1.30 


ALEAD 


abeolute value of I£AD 


-.034 


le37 


AINSTR 


absolute value of jkstr 


AAA 

•009 


.35 


AOONFL 


Abeolute value of OONFL 


-•057 


V A"? 

1*9/ 


AEACE 


Abeolute value of FACE 


•016 


•64 




nigneet degree or principal 


-•322 


O ^ A 

2e69 


PRINEXP 


Experience teaching of Principal 


•055 


4 A^ 

4.01 




amma 4 m *Am4 m4 Minuet 4 

experience in acniinistration 


•Uoo 


6.67 


SEX 


Female equals one 


-1.486 


11.73 




wnite equals one 




O. 11 




(jniiancoa experience 


•UUJ 


•U4 


SB0035 


Total parental involvement 
Econartiic status 


A AC 

•085 


1.90 


SBC037 


•015 


16.27 




jrxecesu score 


.7 J* 


lUo.UJ 


PRESQ 


Pretest score squared 


-.0009 


105.26 


nLDiuU 




— f f 


1 AQ 
XeOO 


offstud 


Clerical/student ratio 


-6.318 


1.09 


TTINSTR 


Teacher's time in instruction 


.580 


5.81 


TTPREP 


Teacher's time in preparation 


.293 


2.77 


TTAEMIN 


Teacher's time in a&ninistration 


-.203 


1.67 


T0VOO1 


Years teaching of teacher 


.011 


1.14 


TQA004 


Highest degree of teacher 


-.715 


4.87 


TO006 


Math courses taken by teacher 


.063 


.91 


TQA012 


Math insexvice 


-.022 


4.16 


SCHADA 


School enrollment 


-.0006 


2.20 


UNION 


Union equals one 


.627 


3.98 



Dependent Variable t Post Test Score 



R-squared _ 
No. of Observations 



14,959 



.55 
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Table 3x Means of Variables at the School Level 



Variable 


Variable Definition 


Mean 


Standard 


Name 






Deviation 


LEAD 


Active Principal Leadership 


-.051 


.707 


INSTR 


Instructional Leadership 


-.025 


.600 


CONFL 


Conflict identified 


.006 


.519 


FACE 


Work wall together 


.090 


1.414 


ALEAD 


absolute value of LEAD 


-.174 


2.929 


AINSTR 


absolute value of INSTR 


-.170 


2.669 


ACONFL 


Absolute value of CONFL 


-.208 


2.769 


AFACE 


Absolute value of FACE 


.001 


3.237 


PDEGREE 


Highest degree of Principal 


2.968 


.270 


PRINEXP 


Experience teaching of Principal 
Experience in administration 


10.473 


5.629 ' 


PEXPAD 


14.814 


7.787 


SEX 


Female equals one 


.514 


.106 


RACE 


White equals one 


.738 


.085 


SBC034 


Childhood Experience 


1.037 


.392 


SBC035 


Total parental involvement 


1.939 


.966 


SBC037 


Economic status 


223.735 


59.426 


PRE 


Pretest score 


29.895 


6.421 


PRESQ 


Pretest score squared 


1842.523 


4822.156 


ADSTUD 


Administrators/ student ratio 


.004 


.003 


QFFSTUD 


Clerical/a cudent ratio 


.019 


.012 


TTINSTR 


Teacher's time in instruction 


4.962 


.604 


TTPREP 


Teacher's time in preparation 


1.417 


.567 


TTAEMIN 


Teacher's time in administration 


.758 


.459 


TQA001 


Years teaching of teacher 


12.422 


6.447 


TQA004 j 


Highest degree of teacher 


2.452 


.424 


TQA006 


Math courses taken by teacher 


.588 


.817 


TQA012 


Math inservice 


6.426 


9.470 


SCHADA 


School enrollment 


433.343 


261.783 


UNION 


Union equals one 


.649 


.478 


POST 


Post test score 


39.537 


. 6.440 



Source: ['Sustaining Effects Study" conducted by the Systems 
Development Corporation, Santa Monica, CA. 
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Table 4: Estimates of the Effect of Principal Variables' on 
Teacher Activities 



Explanatory 
Variable 



Dependent Variable 
Time Teachers spend in: 



Instruction Preparation ftflninistration Inservice 



INTERCEPT 








7 AOQ 


t inn 


— AQQ 
"•W77 




OA7 




JLDoTK 


—1 fYYAtir 
— l.U/«* w 


• UUO 


• uou 


1 79A 


Uxtrxj 




1 1A 


« uuo 


O 9 AO** 


FACE 


•UUb 


— eUX/ 


— • UX4 . 


•••U/^J 


AL£AD 


ni st 


— «UUJ 




• UU7 






• UUX 


•"•UXX 


— 1&A 






— ono 


• UUJ 


— • JOX 


AFTiCE 




• uzz 


• • uuux 


■"• UX7 


QAlAAl 

FQAJJOI 




— iW 




1 7QQ 
X • /DO 


P0KX)2 


-.005 


-.017** 


.004 


.013 




••UUO 


mm nrm 

"■•UUU 


• vaai 


•"• UUQ 


CSV 
OCA 


ACS) 


m m HU m W 


- 502* 




RATE 


.042 


-.277 


.346 


-3.537 


SBC034 


-.169* 


-.042 


.033 


-.965 


SBC037 


.002* 


-.0001 


.0009 


.003 


PRE 


-.004 


.008 


-.002 


-.198 


PKESQ 


-.00001 


-.00001 


-.00001 


.0003 


AD6TUD 


18.247 


-.'750 


-12.154 


-109.220 


OFFSTUD 


-5.500 


3.701 


5.464** 


;2.702 


TQA001 


.012** 


.009* 


.007 


.031 


TO004 


-.043 


.267** 


.048 


.576 


TQA006 


.066 


-.036 


.020 


.002 


SCHAEA 


-.0001 


-.0001 


.0001 


.002 


UNION 


-.235** 


.050 


.020 


-1.102 


R-squared 


.19 


.12 


.11 


.09 


No. of observations 


285 


285 


285 


285 



Note: (*) denotes significance at the 10 percent level: 
(**) denotes significance at the 5 percent level, 
description of variables 1n Table 3. 
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TabU *t Suiwry ut Kaaulr* 



(A) 1)1 r«cr a UnMMunit Indlracr Principal EfUcta on POST 
(Tabla 2) 



VA* 


UAP ♦ 


lNSTfc ♦ 


CUMFL ♦ 


PACK ♦ 


AUAD - 


Alum - 


ACDHPL - 


APACfc - 


MM 




♦ 


♦ 




















VAX 


pgAOUi 
(Id) ♦ 


P<MUU2 
(T Imp) ♦ 


PUtPAD 
(Ad Ixp) ♦ 




SUN 




♦ 


♦ 



V 



(•) Principal Kf focta on Datarmlnaota of POST 
(Tabla A) 



Explanatory Vtrlabla 



Dapaaaaat 

Varlabla 


LEAD ♦ 


INST* ♦ 


CON PL ♦ 


PACK ♦ 


ALtAO - 


AlNtTH - 


A GO* PL - 


APAd - 


PQA001 ♦ 


PQA0O2 ♦ 


PCX PAD ♦ 


Pa rascal 
lovolvaaant 

TTlNSTx 

TTPHIP 
























TTA0HIM 
ravaraa algna 

TQAU12 
lnaarvica 














-10X 











•NOTE: 11 algna eaacad ar ,U5 laval, axcapr «#hara otharvlaa lndleatad. Tha algna naxt co eha varlabla nana danora rha 
hypochaaliad direction of rha alfact. Tha algna In eha box dlracc rha aaelaaead dlracelon of rha affacr* If no algn 
la dlaplayad, than rha aariaaca vaa not algnlf leant ly dlffarant froa iaro« 
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Appendix A: Variables Used in Multivariate Regression Analysis of Growth in 
Student Achievement on Math and Reading Tests 
(Source: Sustaining Effects Study) 



A. STUDENT VA RIABLES 

f 

1) Age 

2) Sex 

3) Race 

4) Economic status: Percentage of the Orshandky Poverty Index, range 
4b-427. 

5) Childhood experience: The exposure of the student to either 
kindergarten, heads tart, summer school, nursery/day care, or preschool, 
range 0-1. 

b) Parent involvement: Participation of the student's parent(s) in any 
school activity: positive or negative, related or unrelated to the 
student, range 0-12* 

7) Exact grade level: An index of student grade and entrance time, range 
1.1-6.3 



B. ACHIEVEMENT MEASURES 

1) Achievement growth: The difference between fall at-level and spring 
below-level Comprehensive Test of Basic Skills for Mathematics (and 
Reading) . 

' I 

2) Pretest: The fall at-level Comprehensive Test of Basic Skills for 
Mathematics (and Reading) . * 

NOTE: The achievement tests were published by McGraw-Hill and 
copyrighted in 1973. 



0. TEACHER CHARACTERISTICS 

1) Experience: Number of years teaching inside and outside the district 

I) Highest degree: 1 - no degree or degree based on fewer than 4 years 

of school 

2 » bachelor's degree 

3 - master's, bachelor's + fifth year preparation, 

or sixth year specialist's degree 

4 - doctor's degree, range 1-4 

J) Course in Math (and reading): Amount of college- level courses in math 
(reading) the teacher has taken, range 0-b. 
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^ 4) Math (reading) inservice: Hours of inservice the teacher has experienced 

in the last three years , range 0-81. 

5) Mode of instruction 

a) individualized approach: A composite of the individualized 
instructional approach of the teacher. The composite is based 

on uses of subgroups , the sameness of time for activities , learning 
sequences, instructional activities, expected rate of progress, 
instructional methods, math content for regular and low-achieving 
students. 

b) class organizations: Teachers were asked to indicate which of the 
following practices are used extensively in their school: 

i) mixing students of different ability levels in 
classes 

ii) assigning students to classes on -the basis of ability 
ill) use of ungraded classes 
iv) flexible scheduling of classes 
v) mixing grades within classes 

• 

b) Contract salary: What is your expected contract salary for teaching in 
this school system this year? Do not include supplements for extra 
services (Mark one): 

Less than $4,000 $16,000-18,999 

$4,000-6,999 $19,000-21,999 

$7,000-9,999 $22,000-24,999 

$10,000-12,999 $25,000 or more 
$13,000-15,999 

7) Time allocation: Teachers were asked to answer the following two 
questions: 



In a tvpical day, how much time do you devote to instructing students? 
Omit recess, lunch, etc. (Mark one.) 

L k>ours 3 hours 4 hours b hours 6 hours 

or 19.B3 or less 

In a typical day, about how much time do you devote to non-instructional 
activities? (Mark the number of hours in each category. Include both 
time at school and time after school.) 
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Other school- 
Preparation Administrative Interactions related non- 
for and clerical with individual instructional 

None instruction tasks parents activities 

1/2 hour or less 

1 hour 

1- 1/2 hours 

2 hours 

2- 1/2 hours 

3 or no re hours 



8) Attitude?: 

a) Educational philosophy 

In your opinion how important is each of the following factors in 
explaining student academic performance? (Mark one column for each factor.) 

Among Among 
The most the more the less The least 
Important important Important Important 

Income level of family 

Home environment 

Ability level of student 

Student motivation 

Adequacy of instructional materials 

Quality of instruction provided 



b) Instructional technique: Teachers were asked to answer the 
following questions: 

Which of the following statements most nearly describes your own approach 
to the use of rewards in the classroom? (Mark one.) 

I try to offer recognition to students primarily when they achieve 
specific objectives. I u*e praise or other rewards mainly to help 
students acquire specific academic skills and social behaviors. 

V 

I try to establish a warm accepting climate for all students, giving them 
praise, affection, and other rewards no matter how well they achieve, or, 
within limits, what they do. 

About how frequently, do you use each of the following techniques to 
handle disruptive classroom behavior? (Mark one column for each procedure.) 

Se ve ral Se ve ral Abou t 

times a times a once a Very in- 

week month month frequently Never 

Isolate the student in the classroom 
Send student to alternate room 
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Provide alternate activity 

Change student's seat 

Send student to higher authority 

c) (antral attitudes concerning school 

how satisfied are you with the way most decisions are made in your 
school? (Hark one.) 

Very Satisfied Somewhat Satisfied Not Satisfied at All 

For each of the statements below concerning this school, indicate your 
general feelings. (Mark one column for each statement.) 



Strongly Strongly 
agree Agree Disagree disagree 
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The school's program are well 

planned and clear 
The school's programs are success* 

fful in meeting students' needs 
Teachers in this school conduct 

effective instruction 
Teachers in this school work 

well together 
This school is a satisfying place 

in which to work 
Staff development training 

provided me is adequate 
1 have sufficient resources 

to carry out effective 

instruction 
Conflicts among individuals are 

identified and faced, 

and not allowed to fester 



d) Teacher assessment of administrative leadership: Response of 
teachers to various questions: 

Which phrase best describes your principal's (or other members of the 
administrative staff) support of your work as a teacher? (Mark one.) 

Very supportive Somewhat unsupportive 

Somewhat supportive Very unsupportive 

Neutral 

Which of the following statements best characterizes your principal's 
role with respect to your school's instructional programs? (Mark one.) 

The principal has a distinct point of view, and actively 
promotes it. 

The principal has a point of view, but lets teachers 
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do as they think best. 

/ Does the principal of your school encourage teachers to try our new 

teaching method? (Hark one.) 

Yes, the principal openly encourages innovation. 

Yes, but only under close supervision. 

No, the principal prefers proven methods. 

The principal neither encourages nor discourages innovation. 

For each of the statements below concerning this school, indicate your 
general feelings: 



Strongly Strongly 
agree Agree Disagree disagree 



The principal provides 
active leadership to 
reading and mathematics 
programs 

Administrators keep 
teachers well informed 



D. PRINCIPAL VARIABLES 

1) Personal Characteristics (Same Variables as Teachers) 
I) Instructional Leadership Variables 

i a) About how frequently do you or your assistant Jbserve classroom 
instruction for regular and low-achieving students for a period 
of ID minutes or more? (Mark one in each column.) 

$ 



Regular Students Low-Achieving Students 



a. Do not observe or do not 
have this type of student 

b. Once a semester 

c. Once a month 

d. Once a week 

e. More than once a week 

f. Daily 



ERLC 



34 

38 



3) Amount of staff development training 



For the following areas, indicate how much staff development training 
(sponsored by the school, district, region, or state) you have received 
within the last three yea^s. Include only formal courses or workshops. 
(Mark one column for each area*) 

i Only training 

1-3 4-6 7-10 * More than was more than 

None hours hours hours 10 hours 3 years ago 



a. Curriculum and 
instruction for 
low-achieving 
students 

b. Leadership 
techniques 

c. School-community 
relations 



d. Needs assessment 
procedures 



e. Program planning 
procedures 



i. Program monitoring * 
and evaluation _ 
procedures 



b) Time principals spend on certain activities related to 
administrative leadership 

how much time have you spent during this school year by participating in 
activities related to curriculum developmet t in your school (e.g. , attending 
workshops, attending local school meetings on curriculum development, etc.) 
in the areas listed below? (Mark one in each column.) 

Reading Math 



a. 


None 


b. 


1-5 hour 8 this year 


c. 


6-10 hours this year 


d. 


11-15 hours this year 


e. 


lb-20 hours this eyar 


f . 


More than 20 hours ,this year 
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4) Principal's attitude toward: 



a) Education 

Which of these statements comes closest to expressing your philosophy? 
(Mark one.) 

a. Most children learn best when lessons are specific, presented 

to the child in small steps, and with a well-planned sequence 
worked out in advance* With this in mind, teachers should 
carefully structure the lessons and experiences of their 
students. 

b. _____ Most child^n learn best by discovering things for themselves. 

Too much structure hinders their natural desire to discover, 
learn, and explore. Teachers should help students, not give^ 
them pre-digested materials. 

^ To what extent do you believe the following are effective in raising the 
achievement of low-achieving students? (Mark one column for each group.) 

v? Moderately Generally 

Effective effective ineffective 

a. Retraining teachers to teach 

low-achieving students _____ 

b. Mixing students of different 

achievement levels in classes mmmmmmm 

c. Having small classes (10 or 

fewer students) • 

d. Increasing the number of 

remedial reading and math 

support personnel * ~ 

e. Increasing numbers of 

counseling and psycho- * 

therapy personnel 

t. Staring remedial education 

at the preschool level _____ 

g. Extensive use of cross- 
age tutoring " 

h. Parent involvement in 
classroom activities 



b) Importance of teacher participation 
In which of the areas listed below is teacher participation very 
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• 



V 

important, important, or unimportant? (Mark one column fbr each decision 
area.) 



Very Important Important Unimportant 



a. Assignment of students 
to classes 

b. Development of student 
grading procedures 

c. Assignment of teachers 
to classes 

d. Selection of basic 
instructional materials 

e. Planning of couise content 

f • Promotion of school/ 
community interaction 



c) Principal's assessment of own leadership (same as teacher's 
assessment; 



E. DISTRICT CHARACTERISTICS 

NOTE: These characteristics were reported by the district superintendent or 
a designated member of his/her staff 

1) Student enrollment K-12 on or about October 1 by school 

1) Staff personnel 

a) official administrative 

i) district administrators 
ii) building administrators 



b) professional 



curriculum specialist 
counselor 

library/ media personnel 
teacher 

psychologist, therapist, etc. 
community relations personnel 
social worker 
other 



c) technical, office, clc.icai 



ERLC 
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student teacher/ teaching assistant/ teaching intern 

teaching aide 

office/clerical 

other 



3) School climate and extent of physical violence: A composite measure was 
constructed by indexing the responses, to the following questions: 

Compared to other schools with which you are familiar, is there much 
vandalism in this school and on the school ground? (Mark one.) 

a. A great deal of vandalism 

b. _____ An average amount of vandalism 

c. _____ Less vandalism than in other schools 

d. No vandalism at this school 

Compared to other schools with which you are familiar, is there much 
physical violence in this school (.e.g, fights among students, attacks on 
teachers, etc.)? (Mark one.) 

a. ____ More than in other schools 

b. About the same as in other schools 

c. _____ Less than in other schools 

d. No violence in this school 



4) Collective Bargaining 

a) Teachers covered by a collective bargaining agreement: 
teachers were considered to be covered by a collective 
bargaining agreement if the superintendent answered 
affirmatively to the following question 

Are elementary school teachers in your district covered by a master 
contract which was bargained by an organization recognized as the bargaining 
agent for the teachers? 



b) Characteristics of the contract 

Which of the following are covered in your master contract with the 
organization recognized as the bargaining unit for the teachers? 

^feacher participation in selection of classroom materials. 

, Preparation time during normal school hours for elementary 
teachers 
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Teachers participation in budgeting 

Teachers participation in program planning and evaluation 
Staffing and class size 

We do have a contract 9 but it does not cover these issues 
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Const ruction of Variables Used in Estimation 
(A) Direct and Unmeasured Indirect Principal Effects on Post 
(1) Joint Perceptions 
LEAD - (PQA0S1 + TQA080)/2; 

active leadership (P) (T) 

INSTR - (PQA008 + PQOOb + 2 * (TQA057)/4; 
math involv. math part (inserv) principal role 

CONFL - (P0.A054 + PQA058 + T0AO83 + 

satis w/ dec (P) confl. ident (P) sat. w/dec (T) 
* FACE - (PQA053 + TQA082); 

work well tog. (P) work well tog. (T) 

(2) Agreement of Perceptions 

ALEAD - [(PQA051 - TQA080) * ABS ( )]; 

AINSTR - [(PQA008 - TQA057) * ABS ( ) + (PQA006 - TQA057) * AB ( )]; 
ACONFL - I (PQA054 - TQA083) * ABS( ) + (P0.AQ58 - TQA084) * ABS( )]/2; 
AFACE - [(PQA053 - TQA082) * ABS ( )]; 



TQA084)/4; 
confl. ident. (T) 
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